Sustaining Engineering Raw Content [DRAFT]

 

 

 

 

 

 

 

 

 

 

 

 

 

Notes to Contributors

Post your raw content under the appropriate heading. Please include a title or the name of the topic and your name and date. Please insert and horizontal line between posts.



 

Architecture and Design [DRAFT]

[Title or Topic]

[Author and Date]

[Content]

 


 

 

[Title or Topic]

[Author and Date]

[Content]

 

 

 

 

 



Configuration Impacts [DRAFT]

 

[Title or Topic]

[Author and Date]

[Content]

 


 

 

[Title or Topic]

[Author and Date]

[Content]

 

 

 

 

 



Environmental Impacts [DRAFT]

 

[Title or Topic]

[Author and Date]

[Content]

 


 

 

[Title or Topic]

[Author and Date]

[Content]

 

 

 

 



Log Analysis and Interpretation [DRAFT]

 

Tools and Utilities for StorNext Performance Troubleshooting

Paul Martin provided the following.

Network Utilities

netstat

Is a command-line tool that displays network connections (both incoming and outgoing), routing tables, and a number of network interface statistics. It is available on Unix, Unix-like, and Windows NT-based operating systems.

When to Use It

It is used for finding problems in the network and to determine the amount of traffic on the network as a performance measurement.

How to Use It

netperf 

This will test the network stack performance of both the remote host and machine running netperf.  This tests how many 1-byte (default, can be increased) packets the two machines can send and respond to per second.  When to Use It

 

More Info
Netperf Homepage

 

 ttcp

Test TCP (TTCP) is a command-line sockets-based benchmarking tool for measuring TCP and UDP performance between two systems.

When to Use It

Particularly useful in DLC environments for load generation and validation or network performance (available on windows and linux)

How to Use It
 

For example
 

DLC server:                                        
ttcp –r –s
DLC client (windows)                       
ttcpw.exe –t –s <ip address of DLC server>

 

After receiving 2048 buffers, ttcp on the DLC server will report network bandwidth information, for example:

 

ttcp-r: 16777216 bytes in 0.193 real seconds = 84775.668 KB/sec +++
ttcp-r: 3285 I/O calls, msec/call = 0.060, calls/sec = 16997.563
ttcp-r: 0.000user 0.036sys 0:00real 15% 0i+0d 0maxrss 0+2pf 3252+0csw

 

More Info

 

ntop (network top)
ntop displays network usage in a fashion similar to top output (also web based version). It can run on linux/UNIX and Windows.

ntop provides a web browser (e.g. netscape) for navigating through ntop traffic information and get a dump of the network status.

ntop is based on libpcap, which provides network statistics collection, security monitoring and network debugging.

 

When to Use It

 

How to Use It


More Info

 ntop Overview
ntop Documentation

 

StorNext Utilities

 

cvadmin
 

who: determine what clients are connected
latency-test all: determine if there is an abnormal delay beteen the MDC and any clients
show long: look for full stripe groups, check multipath config
disks: verify that all disks are showing
disks refresh: when disks might have changed (zoning change, etc.)

When to Use It

How to Use It


fs_scsi
Used for probing raw devices attached to the system, which can. Can be useful in checking for scsi3 reserves. Can be used to quickly verify that tape drives and medium changes are available with "
fs_scsi -p | grep -i ult" (for tape drives) or "
fs_scsi -p | grep -i medi" (for tape robot, "medium changer").

cvlogparser

More Info
Mark_Cookson’s cvlogparser PowerPoint Presentation
Interpret cvlog Fields

Unix Utilities:

free -m 
Shows how much memory is free (-m in megabytes) and how much is used by swap.  Best if swap is 0.

More Info
free Man Page

sar 
System Activity Reporter. Available on a UNIX derivates as standard. On Linux as part of the sysstats rpm package

Useful for monitoring system resource utilization over time.
More Info
sar Man Page
Mac Utilities:

Disk Utility (Mac)
Similar to df and mount for the Mac.

xsanctl (Mac) - Xsan 2 Adim Guide
Used for mounting and unmounting volumes on Xsan.  Use sanConfigChanged option to tell xsan to reread fsnameservers and other config files.  Use disksChanged to cause it to rescan disks.

Windows Utilities

To have linux commands on windows, use the native Win32 GNU Utilities (executables only depending on the Microsoft C-runtime (msvcrt.dll). It provides executables like tail, grep, less, etc. http://unxutils.sourceforge.net/

Perfmon (Windows)

Used for testing performance of Windows applications on local and remote systems.

When to Use It

How to Use It
1. Launch Command Prompt (alternatively, cmd.exe)
2. Enable StorNext perfmon tracing counters
a. Run c:\program files\stornext\bin\cvdbset.exe perf
b. Run c:\program files\stornext\bin\cvdb.exe -P
3. Launch perfmon
4. Launch the application to be tested.

More Info
Windows XP Perfmon Command-line Reference
MSDN Library: How to Monitor System Performance Using Performance Counters
Disk Utilities:

dd

A dd utility that will copy a file from an input source (if=) to an output file (of=).  To test write speeds, specify if=/dev/zero to create a file full of 0's from an input source that won't slow down the writes.  To test read speeds, specify of=/dev/null to read a file and throw away the bytes in a way that won't slow down the read.

Performance Utilities:

A good overview of several kinds of benchmarks can be found on the Linux Benchmark Suite Homepage.

lmdd - http://lmbench.sourceforge.net/man/lmdd.8.html
A dd-like utility that will copy a file, create a file full of 0's, or read a file.  Will give performance figures for the total operation combined when doing a copy, or for the read or write when doing just that.  When used with the wtmax (write-time-max) or rtmax (read-time-max) option it will create a 10 bucket histogram that shows the number of IOs that took the amount of time represented by the bucket.  A setting of rtmax/wtmax=100 is useful.  Can also be used to test tape IO by specifying the /dev device that corresponds to the tape drive.

iometer - http://www.iometer.org/
An I/O subsystem measurement and characterisation tool for single and clustered systems comprising a workload generator and a measurement tool.  It can emulate both disk and network I/O loading.

iozone - http://www.iozone.org/http://linux.die.net/man/1/iozone
Disk benchmarking utility

bonnie - http://www.coker.com.au/bonnie++/
A performance benchmark which targets various aspects of Unix filesystems.

iostat - http://linux.die.net/man/1/iostat
Displays the load average and disk I/O information.

iozone - http://www.iozone.org/http://linux.die.net/man/1/iozone
Disk benchmarking utility

portPerfShow
To view system wide throughput statistics, the portPerfShow command can be run on FC switch.
switch1:admin> portPerfShow
0    1    2    3    4    5    6    7    8    9   10   11   12   13   14   15  Total
---------------------------------------------------------------------------------------------------------------
0    0   21m  28m  31m   0  8.4m   0   28m  21m  31m   0  8.4m   0    0    0  178m

test_toolkit

Perf traces from cvdb
The cvdb perf traces are useful for tracing performance type information on a particular StorNext client. 

It can be used to get the same kind of information that Greg H. gets from the Process monitor. 

To use it, one would follow these basic steps on the client in question:

1.       cvdbset perf              # Set flags for tracing
2.  cvdb –d                   # disable the tracing
3.  cvdb –g > /dev/null       # clear out the trace buffer
4.  cvdb –e                   # enable the tracing
5.  cvdb –g –C –F /tmp/trace  # continuously snap traces into /tmp
6.       Run the application in question
7.  cvdb –d                   # disable the tracingv
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 
When to Use It

As StorNext doesn't use 1-byte packets, this isn't a real-world test, but it gives you a good idea if one machine is better at than another for handling network requests. Typical numbers for Macs are about 4000 and Sunfire boxes about 15000, Linux boxes fall more towards the middle.

How to Use It

 

 

 

 

/opt/netperf/netperf -H remotehost -t TCP_RR

 

 

Test TCP (TTCP) is a command-line sockets-based benchmarking tool for measuring TCP and UDP performance between two systems.


 

 

 

[Title or Topic]

[Author and Date]

[Content]

 

 

 

 



Case Studies [DRAFT]

 

[Title or Topic]

[Author and Date]

[Content]

 


 

 

[Title or Topic]

[Author and Date]

[Content]

 

 

 

 


 Troubleshooting StorNext Performance >

 

 

 

 

 

 

 

 

ttcp



This page was generated by the BrainKeeper Enterprise Wiki, © 2018